Estimated optimal video encoding parameter set systems and methods

ABSTRACT

The systems and methods are configured to efficiently and effectively determine or find an estimated optimal encoding parameter set. In one embodiment, a video encoding parameter set estimation method comprises: performing an offline encoding parameter set characteristic prediction process that determines an estimate of a candidate encoding parameter set characteristic; and performing an encoding parameter set search process that identifies a predicted or estimated optimal video encoding parameter set. The encoding parameter set search process can include applying a constraint to the candidate encoding parameter set characteristic; and determining if candidate encoding parameter set meets an objective, wherein the determining is performed if the constraint is satisfied. The candidate encoding parameter set characteristic can be an estimated encoding time of the candidate encoding parameter set. The objective can be the best video quality out of a plurality of candidate encoding parameter sets.

RELATED APPLICATIONS

This Applications is a continuation of and claims the benefit of andpriority to U.S. patent application Ser. No. 16/857,061 entitled“Estimated Optimal Video Encoding Parameter Set Systems and Methods”filed on Apr. 23, 2020, which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

Numerous areas of business, science, education, and entertainmentutilize video to convey information in various important applications(e.g., medical procedures, vehicle aided operation, financialapplications, etc.). Frequently, utilizing video involves processing andconveying large amounts of information. However, processing andcommunication bandwidth of video systems is typically limited. Sometraditional systems utilize video encoding to compress data whencommunicating and storing the video information. Traditional encodingapproaches can be problematic, expensive, and time consuming. Forexample, conventional video encoding can involve significant costs andconsumption of resources (e.g. resources to perform encoding/decodingoperations, hours of run time, etc).

Electronic systems typically utilize a codec to encode/decode (e.g.,compress/decompress, etc.) video information for communication andstorage. Video encoding parameters typically direct video encodingoperations and have a significant impact on results. Various encodingcharacteristics (e.g., including video quality, encoding time, etc.)vary depending upon specific encoding parameters set selection. In aH.264 encoding scheme there are sequence encoding parameter sets andpicture encoding parameter sets that include information such as picturesize, option coding modes, macro-block to slice group maps, etc.Efficient and effective results often depend upon selecting a good oroptimal video encoding parameter set. There are usually numerousdifferent types and approaches to video encoding and compression.Finding solutions that provide efficient and effective results can bedifficult. Searching encoding parameter sets with a brute force searchtypically requires very large amounts of resources and time. In oneexemplary X.264 with 23 encoding parameters to select from, a videocodec has 104,044,953,600,000 cases to search. As a practical matter itcan be impossible to check every possible encoding parameter combinationthat can be included in an encoding parameter set with a real-timeencoding test. To verify a real-time encoding constraint it needs tocheck encoding time with a candidate encoding parameter set. Traditionalapproaches measure real encoding time for a candidate encoding parameterset. With the assumption of 1 second encoding time per set, it can takemore than 3 million years to check the encoding time of every possiblecandidate.

SUMMARY OF THE INVENTION

The systems and methods are configured to efficiently and effectivelydetermine or find an estimated optimal encoding parameter set. In oneembodiment, a video encoding parameter set estimation system comprisesan offline encoding parameter set characteristic prediction module andan encoding parameter set search module. The offline encoding parameterset characteristic prediction module determines an estimate of encodingtime of a candidate encoding parameter set from an inputted videostream. The encoding parameter set search module identifies an estimatedoptimal video encoding parameter set. In one embodiment, the encodingparameter set search module is configured to: apply a time constraint tothe encoding time of the candidate encoding parameter set, and basedupon the time constraint being satisfied, determine if candidateencoding parameter set meets a video quality objective. The encodingparameter set search module can also establish the candidate encodingparameter set as an estimated optimal parameter set when the videoquality objective is met.

In one embodiment, the video quality objective is for the candidateencoding parameter set to have better video quality than a plurality ofother candidate encoding parameter sets. The offline encoding parameterset characteristic prediction module can include a reference setmeasurement module, wherein the reference set measurement module isconfigured to measure an encoding time of a reference encoding parameterset. The reference set measurement module can be configured to measure avisual quality of a reference encoding parameter set. The offlineencoding parameter set characteristic prediction module can include aspecific value difference measurement module configured to measure anencoding time difference of a specific value encoding parameter set. Thespecific value difference measurement module is configured to measure avideo quality difference of a specific value encoding parameter set. Theencoding parameter set search module utilizes candidate encodingparameter set information and information from the offline encodingparameter set characteristic differential module.

In one embodiment, the encoding parameter set search module isconfigured to: compare an estimated encoding time to a time constraint;and conditioned upon satisfying the time constraint, a further searchanalysis of a video quality characteristic of the video encodingparameter set is performed based upon the objective. The encodingparameter set search module can be configured to establish a candidateencoding parameter set with a video quality characteristic closet to theobjective as the predicted or estimated optimal video encoding parameterset.

In one embodiment, a video encoding parameter set estimation methodcomprises: performing an offline encoding parameter set characteristicprediction process that determines an estimate of a candidate encodingparameter set characteristic; and performing an encoding parameter setsearch process that identifies a predicted or estimated optimal videoencoding parameter set. The encoding parameter set search process caninclude applying a constraint to the candidate encoding parameter setcharacteristic; and determining if candidate encoding parameter setmeets an objective, wherein the determining is performed if theconstraint is satisfied. The candidate encoding parameter setcharacteristic can be an estimated encoding time of the candidateencoding parameter set. The objective can be the best video quality outof a plurality of candidate encoding parameter sets.

In one embodiment, the offline encoding parameter set characteristicprediction process includes: an offline encoding characteristicdifferential process that determines differences in a characteristic ofa reference parameter set and a characteristic of a specific valueparameter set; and an encoding parameter set summation process that addsoffline encoding characteristic differential values together. Theoffline encoding characteristic differential process can include ananalysis of a characteristic based upon controlled changes to anencoding parameter of the specific value parameter set. The controlledchanges make an encoding parameter in the specific value parameter setencoding different than the encoding parameters in the referenceparameter set. In one embodiment, one encoding parameter is changed inthe specific value parameter set to be different than the encodingparameters of the reference parameter set while the remaining parametersin both the specific value parameter set and reference parameter set arethe same. In one exemplary implementation, a difference between theencoding time of the reference parameter set and the encoding time of aspecific value parameter set is determined. In one exemplaryimplementation, a difference between the video quality of the referenceparameter set and the encoding time of a specific value parameter set isdetermined. If an assigned characteristic value does not meet the searchconstraint the encoding parameter set associated with assigned value canbe removed from consideration as an estimated optimal encoding parameterset. An encoding parameter search process can include dynamicprogramming.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present technology are illustrated by way of exampleand not by way of limitation, in the figures of the accompanyingdrawings and in which like reference numerals refer to similar elementsand in which:

FIG. 1 is a block diagram of an estimated optimal video encodingparameter set method in accordance with one embodiment.

FIG. 2 is a block diagram of an estimated optimal video encodingparameter set system in accordance with one embodiment.

FIG. 3 is a block diagram of video encoding time prediction method inaccordance one embodiment.

FIG. 4 is a flow chart of offline time prediction method in accordancewith one embodiment.

FIG. 5 is a flow chart of an exemplary in-line encoding time predictionprocess in accordance with one embodiment.

FIG. 6 is a block diagram of a video compression time prediction systemin accordance with one embodiment.

FIG. 7 is a block diagram of a system operations in accordance with oneembodiment.

FIG. 8 is a block diagram of an exemplary encoder in accordance with oneembodiment.

FIG. 9 is a block diagram of an exemplary computing system forprocessing content in accordance with one embodiment.

FIG. 10 is a block diagram of an exemplary processing core in accordancewith one embodiment.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the embodiments of the presenttechnology, examples of which are illustrated in the accompanyingdrawings. While the present technology will be described in conjunctionwith these embodiments, it will be understood that they are not intendedto limit the technology to these embodiments. On the contrary, theinvention is intended to cover alternatives, modifications andequivalents, which may be included within the scope of the invention asdefined by the appended claims. Furthermore, in the following detaileddescription of the present technology, numerous specific details are setforth in order to provide a thorough understanding of the presenttechnology. However, it is understood that the present technology may bepracticed without these specific details. In other instances, well-knownmethods, procedures, components, and circuits have not been described indetail as not to unnecessarily obscure aspects of the presenttechnology.

Some embodiments of the present technology which follow are presented interms of routines, modules, logic blocks, and other symbolicrepresentations of operations on data within one or more electronicdevices. The descriptions and representations are the means used bythose skilled in the art to most effectively convey the substance oftheir work to others skilled in the art. A routine, module, logic blockand/or the like, is herein, and generally, conceived to be aself-consistent sequence of processes or instructions leading to adesired result. The processes are those including physical manipulationsof physical quantities. Usually, though not necessarily, these physicalmanipulations take the form of electric or magnetic signals capable ofbeing stored, transferred, compared and otherwise manipulated in anelectronic device. For reasons of convenience, and with reference tocommon usage, these signals are referred to as data, bits, values,elements, symbols, characters, terms, numbers, strings, and/or the likewith reference to embodiments of the present technology.

It should be borne in mind, however, that these terms are to beinterpreted as referencing physical manipulations and quantities and aremerely convenient labels and are to be interpreted further in view ofterms commonly used in the art. Unless specifically stated otherwise asapparent from the following discussion, it is understood that throughdiscussions of the present technology, discussions utilizing the termssuch as “receiving,” and/or the like, refer to the actions and processesof an electronic device such as an electronic computing device thatmanipulates and transforms data. The data is represented as physical(e.g., electronic) quantities within the electronic device's logiccircuits, registers, memories and/or the like, and is transformed intoother data similarly represented as physical quantities within theelectronic device.

In this application, the use of the disjunctive is intended to includethe conjunctive. The use of definite or indefinite articles is notintended to indicate cardinality. In particular, a reference to “the”object or “a” object is intended to denote also one of a possibleplurality of such objects. The use of the terms “comprises,”“comprising,” “includes,” “including” and the like specify the presenceof stated elements, but do not preclude the presence or addition of oneor more other elements and or groups thereof. It is also to beunderstood that although the terms first, second, etc. may be usedherein to describe various elements, such elements should not be limitedby these terms. These terms are used herein to distinguish one elementfrom another. For example, a first element could be termed a secondelement, and similarly a second element could be termed a first element,without departing from the scope of embodiments. It is also to beunderstood that when an element is referred to as being “coupled” toanother element, it may be directly or indirectly connected to the otherelement, or an intervening element may be present. In contrast, when anelement is referred to as being “directly connected” to another element,there are not intervening elements present. It is also to be understoodthat the phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting.

The systems and methods help efficiently and effectively determine orfind an estimated optimal encoding parameter set. In one embodiment,encoding parameter sets are searched to select or determine an estimatedoptimal video encoding parameter set. The search to determine theestimated optimal encoding parameter set is made with a video encodingtime constraint. The time constraint reduces the search region in theencoding parameter space which makes it possible to relatively quicklyfind an estimated optimal encoding parameter set. In one exemplaryimplementation, an estimated optimal encoding parameter set can bedetermined within seconds (unlike approaches that may takes years andare impractical). In one embodiment, the search is directed at findingan encoding parameter set whose estimated or predicted encoding time isless than a time constraint and has a particular characteristic. Theparticular characteristic can be a certain video quality characteristic.In one exemplary implementation, the search is directed to finding thehighest peek signal noise ratio (PSNR) encoding parameter set. Estimatedoptimal encoding parameter set selection or determination can includedynamic programming based on the video quality, an encoding parameterset predicted or estimated encoding time, and an encoding timeconstraint. In one embodiment, the video quality is a prediction orestimation of the video quality.

In one embodiment, video encoding time prediction or estimation isimplemented. A reference encoding parameter set is defined orestablished and used to measure an encoding time which is used as areference encoding time. Iterations of measuring encoding times for thereference encoding parameter set are performed in which one of theparameters in the reference parameter set is changed for an iterationand the other parameters from the reference encoding parameter set areunchanged. The difference between the reference encoding time and anindividual parameter change encoding time is determined. The individualparameter change encoding times corresponding to parameters in acandidate set are added or summed together and the result is consideredthe predicted encoding time. The encoding time of video content by acodec or encoder with a specific encoding parameter set can be predictedor estimated without performing actual/real encoding with the specificencoding parameter set. In one exemplary implementation, the predictedencoding time can be utilized as the time constraint for the estimatedoptimal encoding parameter set search. In one embodiment, a H.264 orX2.64 medium preset is used as a reference encoding parameter set. AH.264 or X2.64 medium preset can be for determining or setting timeencoding time constraints.

In one embodiment, a predicted or estimated optimal encoding parameterset is an approximation or estimation of an optimal encoding parameterset. An estimated optimal encoding parameter set is not necessarily themaximum optimal encoding in all respects. It may be possible that aparticular encoding parameter set may have aspects or characteristicsthat are better in some respects than the estimated optimal encodingparameter set. However, as a general proposition determining or findingthat particular encoding parameter set can take significantly longerthan the estimated optimal encoding parameter set and can beimpractical. Selection of an estimated video encoding parameter set caninvolve differences or trade offs in results. In one exemplaryimplementation, a video encoding parameter set that has a rapid encodingtime may not have a high video quality, and a video encoding parameterset that has a slow encoding time may have a high video quality.

FIG. 1 is a block diagram of an estimated optimal video encodingparameter set method 100 in accordance with one embodiment.

In block 110, an offline encoding parameter set characteristicprediction process is performed. The offline encoding parameter setcharacteristic prediction process determines a predicted or estimatedencoding parameter set characteristic. There can be various encodingparameter set characteristics (e.g., encoding time, video quality etc.)that are predicted or estimated. In one embodiment, an offline encodingparameter set characteristic prediction process includes an offlineencoding characteristic differential process and an encoding parameterset summation process.

The offline encoding characteristic differential process determinesdifferences in a characteristic of a reference parameter set and acharacteristic of a specific value parameter set. In one embodiment, thedifference in a characteristic (e.g., encoding time, video quality,etc.) of the reference parameter set and the specific value parameterset are due to a difference in an encoding parameter between the twosets. The analysis of the characteristic (e.g., visual quality, encodingtime, etc.) is based upon controlled changes to an encoding parameter ofthe specific value parameter set. The controlled changes make anencoding parameter in the specific value parameter set encodingdifferent than the encoding parameters in the reference parameter set.In one exemplary implementation, one encoding parameter is changed inthe specific value parameter set to be different than the encodingparameters of the reference parameter set (the remaining parameters inboth the specific value parameter set and reference parameter set arethe same). There can be multiple characteristics (e.g., visual quality,encoding time, etc.) that are analyzed. In one embodiment, thedifference between the encoding time of the reference parameter set andthe encoding time of a specific value parameter set is determined. Inone embodiment, the difference between the video quality of thereference parameter set and the encoding time of a specific valueparameter set is determined.

The encoding parameter set summation process adds offline encodingcharacteristic differential values together. The offline encodingcharacteristic differential values come from the offline encodingcharacteristic differential process. In one embodiment, offline encodingcharacteristic differential values corresponding to parameters in acandidate encoding parameter set are added together. In one exemplaryimplementation, offline encoding characteristic differential encodingtime values associated with parameters that are included in a candidateencoding parameter set are added together and the summation total orresult is utilized as a predicted or estimated encoding time for thecandidate encoding parameter set.

In block 120, an encoding parameter set search process is performed. Inone embodiment, the encoding parameter search identifies a predicted orestimated optimal video encoding parameter set. The encoding parameterset search process utilizes candidate encoding parameter set informationand accessing information from the offline encoding characteristicdifferential process in block 110. In one embodiment, a predicted orestimated characteristic value (e.g., from block 110, etc.) is assignedto a characteristic of the candidate encoding parameter set. In oneembodiment, the predicted or estimated encoding time characteristicvalue from block 110 is assigned to the candidate encoding parameterset. A search constraint is applied to a candidate encoding parameterset characteristic. The assigned characteristic value is compared to asearch constraint. A search constraint can be associated with variouscharacteristics (e.g., encoding time, video quality, etc.) associatedwith a candidate encoding parameter set.

If the assigned characteristic value does not meet or is outside thesearch constraint the encoding parameter set associated with assignedvalue is removed from consideration as an estimated optimal encodingparameter set. In one embodiment, removing an encoding parameter setreduces the search field or range. Removing an encoding parameter setfrom the search field reduces time and resources expended on furthersearch/analysis of other aspects (e.g., video quality, real-timeencoding, etc.) of the removed encoding parameter set.

If the assigned characteristic value does meet or is within the searchconstraint, the further searching/analysis of encoding parameter setassociated with assigned value is performed as a potential estimatedoptimal encoding parameter set. In one embodiment, a furthersearch/analysis of a characteristic of the video candidate encodingparameter set is performed based upon an objective. The candidateencoding parameter set with a searched/analyzed characteristic closet tothe objective is identified as the predicted or estimated optimal videoencoding parameter set. In one embodiment, the characteristic is videoquality associated with the encoding parameter set and the objective isto determine or find a parameter set closest to a particular videoquality. In one exemplary implementation, the objective is to determineor find a parameter set with the best video quality.

In one embodiment, an encoding parameter search process includes dynamicprogramming. The dynamic programming can be performed with respect tosearch constraints and predicted or estimated characteristic values ofthe candidate encoding parameter set. The dynamic programming eliminatesencoding parameter sets that do not meet a search constraint. In oneexemplary implementation, encoding parameter sets that do not meet anencoding time constraint are eliminated. The dynamic programmingcontinues with remaining encoding parameter sets to determine or find anencoding parameter set that meets or satisfies an objective. A candidateencoding parameter set that meets or satisfies the objective or limit isidentified as the predicted or estimated optimal video encodingparameter set. In one exemplary implementation, an encoding parameterset that allows a particular video quality (e.g., best, above athreshold, etc.) to be achieved is selected as the predicted orestimated optimal encoding parameter set.

FIG. 2 is a block diagram of an estimated optimal video encodingparameter set system 200 in accordance with one embodiment. Optimalvideo encoding parameter set system 200 includes offline encodingparameter set characteristic prediction module 210 and encodingparameter set search module 220. In one embodiment offline encodingparameter set characteristic prediction module 201 is configured todetermine an estimate of encoding time of a candidate encoding parameterset from an inputted video stream. Encoding parameter set search module220 is configured to identify an estimated optimal video encodingparameter set. In one embodiment, the encoding parameter set searchmodule 220 is configured to: 1) apply a time constraint to the encodingtime of the candidate encoding parameter set; 2) based upon the timeconstraint being satisfied, determining if the candidate encodingparameter set meets a video quality objective, and 3) establish thecandidate encoding parameter set as an estimated optimal parameter setwhen the video quality objective is met.

In one embodiment, offline encoding parameter set characteristicprediction module 210 includes reference set measurement module 211 andspecific value difference measurement module 212. In one embodiment,offline characteristic prediction module 210 performs an offlineencoding characteristic differential process (e.g., similar to block110, etc.). Measurement module 211 measures a characteristic (e.g.,encoding time, visual quality, etc.) of a reference encoding parameterset. Specific value difference measurement module 212 measures acharacteristic (e.g., encoding time, visual quality, etc.) difference ofa specific value encoding parameter set. In one embodiment, an encodingtime difference between utilization of the reference encoding parameterset and the specific value encoding parameter set is measured. Encodingparameter set search module 220 performs an encoding parameter setsearch process. The encoding parameter set search module 220 utilizescandidate encoding parameter set information and information from theoffline encoding characteristic differential module 210. A predicted orestimated characteristic of the encoding parameter set is compared to asearch constraint. If the assigned characteristic value does meet or iswithin the search constraint, a further search/analysis of acharacteristic of the video encoding parameter set is performed basedupon an objective. The candidate encoding parameter set with asearched/analyzed characteristic closet to the objective is identifiedas the predicted or estimated optimal video encoding parameter set. Inone embodiment, encoding parameter set search module 220 performs anencoding parameter set search process similar to block 120 (e.g., withdynamic programming, etc.).

There are a lot of encoding parameters that affect codec performance.Encoding parameters can include profile and level parameters, targetbitrate, frame rate, resolution, number of reference frames, largestcoding unit (LCU) size, group of picture (GOP) structure (e.g., length,number of bidirectional predicted picture (B) frame in GOP, etc.),entropy coding mode, CAVLC/CABAC, motion search range/shape (e.g.,16×16, 8×8, etc.), intra/inter encoding partition size, intra-codedpicture (I), B, and predicted picture (P) frame initial quantizationparameter (QP), node decisions, bit ratio among I, B and P frames, andso on. By optimizing encoding parameters of the target video or scene,more efficient video compression is achieved.

In one embodiment, offline operation can be performed by an optionalseparate characteristic prediction system 230. Separate characteristicprediction system 230 can perform a characteristic prediction orestimation for a candidate encoding parameter set characteristic. In oneexemplary implementation, separate characteristic prediction system 230performs a video encoding time prediction method.

FIG. 3 is a block diagram of video encoding time prediction method 300in accordance one embodiment. In one embodiment, video encoding timeprediction method 300 is similar to operations of offline encodingparameter set characteristic prediction process 110. It is appreciatedthat portions of video encoding time prediction method 300 can beperformed offline and portions can be performed inline.

In block 310, an offline encoding time prediction process is performed.In one embodiment, the offline encoding time prediction processdetermines or generates encoding time information corresponding toparameters in an encoding parameter set. In one exemplaryimplementation, the offline encoding time prediction process determinesdifferences in encoding times of a reference parameter set and acharacteristic of a specific value parameter set. In one exemplaryimplementation, the offline encoding time prediction process candetermine multiple encoding time differences, where each one of theencoding time differences is associated with a change in a differentencoding parameter.

In block 320, an in-line encoding time prediction process is performed.In one embodiment, the in-line encoding time prediction process adds orsums multiple encoding time differences from block 310 that correspondto encoding parameters of a candidate encoding parameter set. The totalor result is utilized or assigned as a predicted or estimated encodingtime for the candidate encoding parameter set.

FIG. 4 is a flow chart of offline time prediction method 400 inaccordance with one embodiment. In one embodiment, offline timeprediction method 400 is similar to the offline time prediction processperformed in block 310.

In block 411, a reference encoding parameter set is defined. In oneembodiment, the reference encoding parameter set is consistent with anencoding standard (e.g., H.264, etc.). The reference encoding parameterset can be based upon an implementation of an encoding standard. In oneexemplary implementation, the reference encoding parameter set isconsistent with the X.264 encoding parameter set. In one embodiment, aH.264 or X2.64 medium preset is used as a reference encoding parameterset.

In block 412, an encoding time of the reference encoding parameter setis measured. The measured time can be used as a reference encoding time.

In block 413, one parameter at a time or iteration is changed in thereference encoding parameter set.

In block 414, the encoding time with the changed parameter is measured.The measured time can be used as a changed parameter encoding time.

In block 415, the difference between the changed parameter encoding timeand the reference encoding time is determined. The difference can beused as a changed parameter difference encoding time.

In block 416, an indication of the changed parameter along with thecorresponding changed parameter difference encoding time is forwarded.The changed parameter difference encoding time information can beforwarded to memory for storage. In one exemplary implementation thechanged parameter difference encoding time information is forwarded toan encoding time prediction process (e.g., 320, 500, etc.) or encodingtime prediction module (e.g., 620).

In block 417, a determination is made if each parameter in the referenceparameter set been changed. If each parameter in the reference parameterset has not been changed the process returns to step 113 and anotherparameter is changed is step 113. If each parameter in the referenceparameter set has been changed the process proceeds to step 119.

In block 419, the process ends.

FIG. 5 is a flow chart of an exemplary in-line encoding time predictionprocess 500 in accordance with one embodiment. In one embodiment,in-line encoding time prediction method 500 is similar to the in-lineencoding time prediction process performed in block 320.

In block 521, a candidate encoding parameter set is accessed.

In block 522, indications of changed parameters along with correspondingrespective changed parameter difference encoding times are accessed foreach parameter in the candidate encoding parameter set. In oneembodiment, changed parameter difference encoding times are accessedfrom results of offline time prediction method 400.

In block 523, the accessed changed parameter difference encoding timesfrom block 522 are summed.

In block 524, the total of the summed changed parameter differenceencoding times is utilized as the predicted or estimated encoding timefor the candidate encoding parameter set.

FIG. 6 is a block diagram of a video compression time prediction system600 in accordance with one embodiment. Offline time prediction module610 is configured to provide offline encoding time predictioninformation. In one embodiment, offline time predictions module 610 issimilar to separate characteristic prediction system 230.

In one embodiment, offline time predictions module 610 includes offlinereference encoding time module 611 and offline specific value encodingtime module 612. Offline reference encoding time module 611 isconfigured to perform measurement of a reference encoding parameter setencoding time. Offline reference encoding time module 611 forwards themeasurement of the reference encoding parameter set encoding timeinformation to offline specific value encoding time module 612. Offlinespecific value encoding time module 612 is configured to performmeasurement of an encoding time difference associated with a changed orspecific value encoding parameter in a difference of a specific valueencoding parameter set. Offline time prediction module 610 can forwardan encoding time difference associated with each changed encodingparameter to a memory for storage. In one exemplary implementation,offline time prediction module 610 forwards the encoding time differencefor each encoding parameter to encoding time prediction module 620.Offline time prediction module 610 can forward encoding time differencesto encoding time prediction module 620 for each encoding parameter in acandidate encoding set.

Encoding time prediction module 620 predicts or estimates an encodingtime. In one embodiment, encoding time prediction module 620 accesses acandidate encoding parameter set and the encoding time difference forcorresponding encoding parameters. Encoding time prediction module 620can access an encoding time difference for an encoding parameter from amemory. In one exemplary implementation, encoding time prediction module620 can access an encoding time difference for each encoding parameterfrom offline time prediction module 610. The encoding time predictionmodule 620 sums the accessed encoding time differences. The total orresults of the summed accessed encoding time differences is utilized asan predicted or estimated encoding time for a candidate encodingparameter set. In one embodiment, the encoding time prediction module620 forwards the predicted or estimated encoding time associated with acandidate encoding parameter set to an encoding parameter set searchmodule 220.

FIG. 7 is a block diagram of a system 700 operations in accordance withone embodiment. System 700 performs initialization operations 710,offline operations 720, and encoding parameter set search operations730. Initialization operations 710 initializes a system forimplementation of operations associated with determining an estimatedoptimal encoding parameter set. In one embodiment, initializationoperations 710 establish a characteristic constraint onsearching/analysis of the encoding parameter sets. The characteristicconstraint can be an encoding time constraint on the encoding parametersets. Offline operations 720 can implement an offline encoding timeprediction process. The offline encoding time prediction process can besimilar to operations in block 310. The encoding parameter set searchoperations 730 identify or determine a predicted or estimated optimalvideo encoding parameter set. The encoding parameter set searchoperations 730 utilizes results from initialization operations 710 andoffline operations 720. In one embodiment, parameter set searchoperations 730 also access and utilize candidate encoding parameter setinformation. In one exemplary implementation, parameter set searchoperations 730 access a predicted or estimated encoding time for acandidate parameter set. In one embodiment, a time constraint is appliedto the encoding time of the candidate encoding parameter set. Based uponthe time constraint being satisfied, determining if the candidateencoding parameter set meets a video quality objective. Establishing thecandidate encoding parameter set as an estimated optimal parameter setwhen the video quality objective is met. In one embodiment, if theencoding time meets or satisfies the encoding time constraint, theencoding parameter set search operations 730 determine if a predicted orestimated video quality for the candidate encoding parameter set isbetter than other candidate encoding parameter sets. The encodingparameter set search operations 730 can be similar to encoding parameterset search process operations in block 120.

In one embodiment, a predicted or estimated characteristic value (e.g.,from block 110, etc.) is assigned to a characteristic of the candidateencoding parameter set. In one embodiment, a predicted or estimatedencoding time characteristic value is assigned to the candidate encodingparameter set. The assigned characteristic value is compared to a searchconstraint.

In one embodiment, offline operations 720 include reference operations721, encoding time difference operations 722, and video qualitydifference operations 723. Reference operations 721 measure the encodingtime of a reference parameter set. In one embodiment, referenceoperations 721 also measures the video quality of a reference parameterset. Coding time difference operations 722 measure encoding timedifferences of each encoding parameter with a changed correspondingrespective specific value while preserving the other parameters to bethe same as the parameter values of the reference encoding parameterset. Coding time difference operations 722 can implement a videoencoding time prediction method. The video encoding time predictionmethod can be similar to operations in block 300. Video qualitydifference operations 723 measure video quality differences of eachencoding parameter with a changed corresponding respective specificvalue, while preserving the other parameters to be the same as theparameter values of the reference encoding parameter set. Video qualitydifference operations 723 can be similar to video quality operations ofblock 110.

In one embodiment, found optimal encoding parameter set increases almost1 dB compared with default parameter set in X264. In one embodiment, anestimated optimal encoding parameter set is utilized to encode video forvideo applications.

Table 1 illustrates exemplary results of encoding time predictionresults

TABLE 1 Contents Prediction Real-Time Encoding First video game 38 33application Second video game 47 45 Application

Table 2 illustrates exemplary results of applying First video gameapplication

TABLE 2 PSNR (Peek Signal Encoding Encoding Option to Noise Ratio)Time(sec) medium 35.44 38 Estimated Optimal 36.44 33 Parameter Set

Table 3 illustrates exemplary results of applying Second video gameapplication

TABLE 3 PSNR (Peek Signal to Encoding Encoding Option Noise Ratio)Time(sec) medium 40.29 47 Estimated Optimal 42.09 45 Parameter Set

FIG. 8 is a block diagram of an exemplary encoder 800 in accordance withone embodiment. In one embodiment, encoder 800 can implement portions ofan estimated optimal video encoding parameter set method (e.g., 100,300, 400, 500, etc.). In one exemplary implementation, encoder 800 caninclude portions of an optimal video encoding parameter set system(e.g., 200, 600, 700, etc.). The encoder 800 may be referred to hereinas an image processing element or as an image processor. In oneembodiment, the encoder 800 can be implemented with separate dedicatedhardware. The encoder 800 can include firmware. In one embodiment, theencoder 800 can be implemented using the processor 902 of FIG. 9. Usingthe encoder 800, the data in the RGB domain is converted into data inthe YCrCb color space. A frequency domain transform (e.g., DCT) isperformed to convert the YCbCr data into the frequency domain. Thefrequency domain data is quantized and then entropy encoded. Theresulting encoded image data can be transmitted (e.g., to a decoder, toa memory for storage, etc.).

FIG. 9 is a block diagram of an exemplary computing system 900 forprocessing content in accordance with one embodiment. In one embodiment,computing system 900 can implement portions of an estimated optimalvideo encoding parameter set method (e.g., 100, 300, 400, 500, etc.). Inone exemplary implementation, computing system 900 can include portionsof an optimal video encoding parameter set system (e.g., 200, 600, 700,etc.). The computing system 900 can include processors 902, memory 904and encoder 901. The encoders 901 can be implemented in separatehardware, or in software executing on the processes 902. In oneimplementation, the computing system 900 can be a server computer, adata center, a cloud computing system, a stream service system, aninternet service provider system, a cellular service provider system, orthe like. The processors 902 can be central processing units (CPU),graphics processing units (GPU), neural processing units (NPU), vectorprocessors, memory processing units, or the like, or combinationsthereof. In one implementation, a processor 902 can includecommunication interfaces, such as peripheral component interface (PCIe4)921 and inter-integrated circuit (I²C) interface 933, an on-chip circuittester, such as a joint test action group (JTAG) engine 923, a directmemory access engine 924, a command processor (CP) 925, and cores931-934. The cores 931-934 can be coupled in a direction ring busconfiguration.

Referring still to FIG. 9, the cores 931-934 can execute sets ofcomputing device executable instructions to perform functions oroperations including, but not limited to, outputting an estimatedoptimal encoder parameter set 970. In one embodiment, the operations aresimilar to the processes described herein (e.g., 100,300, 400, 500,etc.). The functions can be performed on individual core 931-934, can bedistributed across a plurality of cores 931-934, can be performed alongwith other functions on cores, and or the like.

The encoding parameter set estimator module 950 can be configured toestimate an optimal encoding parameter set. In one embodiment, theoperations are similar to the modules in systems described herein (e.g.,200, 600, 700, etc.). In one embodiment, encoding parameter setestimator module 9950 estimates an optimal encoding parameter setsimilar to the methods described above with reference to FIGS. 1 and 3.The processors 902 can output encoding parameters 970. The encoder 901can convert the video content 980 to an encoded video content 990 usingthe encoding parameters 970.

FIG. 10 is a block diagram of an exemplary processing core 1000 inaccordance with one embodiment. In one embodiment, exemplary processingcore can implement portions of an estimated optimal video encodingparameter set method (e.g., 100, 300, 400, 500, etc.). In one exemplaryimplementation, exemplary processing core can include portions of anoptimal video encoding parameter set system (e.g., 200, 600, 700, etc.).The processing core 1000 can include a tensor engine (TE) 1010, apooling engine (PE) 1015, a memory copy engine (ME) 1020, a sequencer(SEQ) 1025, an instructions buffer (IB) 1030, a local memory (LM) 1035,and a constant buffer (CB) 1040. The local memory 1035 can bepre-installed with model weights and can store in-use activationson-the-fly. The constant buffer 1040 can store constant for batchnormalization, quantization and the like. The tensor engine 1010 can beutilized to accelerate fused convolution and or matrix multiplication.The pooling engine 1015 can support pooling, interpolation,region-of-interest and the like operations. The memory copy engine 1020can be configured for inter- and or intra-core data copy, matrixtransposition and the like. The tensor engine 1010, pooling engine 1015and memory copy engine 1020 can run in parallel. The sequencer 1029 canorchestrate the operation of the tensor engine 1010, the pooling engine1015, the memory copy engine 1020, the local memory 1039, and theconstant buffer 1040 according to instructions from the instructionbuffer 1030. The processing core 1000 can provide video coding efficientcomputation under the control of operation fused coarse-grainedinstructions. A detailed description of the exemplary processing unitcore 1000 is not necessary to an understanding of aspects of the presenttechnology, and therefore will not be described further herein.

In one embodiment, an estimation or prediction of the optimal parameterset is made. In one exemplary implementation, an assumption is made thatthe quality contribution of each encoding parameter is linearlyindependent. With encoding time prediction algorithm, searching encodingparameter space with dynamic programming reduces searching time. In oneembodiment, instead of performing real-time encoding, the encoding timeis predicted based on the off line or test data, thereby encoding timeof any combination of encoding parameter set could be predicted inreal-time.

In one embodiment, an estimated optimal video encoding parameter setsystem and method can be utilized to compliment other video codec tuningapproaches. Content adoptive video compression technology is a state ofthe art solution which attempts to find optimized bitrate, resolution,and frame rate depending on the predefined genre or scene. The contentadoptive compression can be based upon utilization of an artificialintelligence (AI) assisted video codec. An estimated optimal videoencoding parameter set system and method can be utilized to determineother encoding parameters such as number of b frames, motion estimationmethod, and so on. Thereby more optimal compression efficiency can beachieved.

Traditionally, a heuristic approach is used to find out encoding time,therefore dynamic parameter control cannot be used in a conventionalreal-time encoding environment. In one embodiment, instead of performingreal-time encoding for each particular encoding parameter set, theestimated optimal encoding parameter set method predicts or estimatesaspects of encoding times offline. The encoding time can be based on thea reference parameter set and differences with one encoding parameterchanged at a time (e.g., test data, off-line results, etc.), therebyencoding time of encoding parameter sets with combinations of encodingparameters can be predicted or estimated in real-time. In oneembodiment, search constraints can be utilized to eliminate somecandidate encoding sets without performing additional processing. In oneembodiment, performing some estimated optimal encoding set selectionprocesses offline and using search constraints to reduce/eliminatepossible candidate sets enables determination of an estimated optimalencoding parameter set, that would otherwise be impossible orimpractical utilizing traditional or conventional approaches.

Aspects of the present technology can advantageously estimate an optimalencoding parameter set. The estimated optimal encoder parameter set canadvantageously increase the compression efficiency of the encoder.Aspects of the present technology can also advantageously eliminate theneed for a brute force determination of the optimal encoding parameterset, thereby reducing computational complexity.

The foregoing descriptions of specific embodiments of the presenttechnology have been presented for purposes of illustration anddescription. They are not intended to be exhaustive or to limit thepresent technology to the precise forms disclosed, and obviously manymodifications and variations are possible in light of the aboveteaching. The embodiments were chosen and described in order to bestexplain the principles of the present technology and its practicalapplication, to thereby enable others skilled in the art to best utilizethe present technology and various embodiments with variousmodifications as are suited to the particular use contemplated. It isintended that the scope of the invention be defined by the claimsappended hereto and their equivalents.

What is claimed is:
 1. A video encoding parameter set estimation systemcomprising: an offline encoding parameter set characteristic predictionmodule that determines an estimate of encoding time of a candidateencoding parameter set from an inputted video stream; and an encodingparameter set search module that identifies an estimated optimal videoencoding parameter set, wherein the encoding parameter set search moduleis configured to: apply a time constraint to the encoding time of thecandidate encoding parameter set; based upon the time constraint beingsatisfied, determining if the candidate encoding parameter set meets avideo quality objective, and establishing the candidate encodingparameter set as an estimated optimal parameter set when the videoquality objective is met.
 2. The video encoding parameter set estimationsystem of claim 1, wherein the video quality objective is for thecandidate encoding parameter set to have better video quality than aplurality of other candidate encoding parameter sets.
 3. The videoencoding parameter set estimation system of claim 1, wherein the offlineencoding parameter set characteristic prediction module includes areference set measurement module, wherein the reference set measurementmodule is configured to measure an encoding time of a reference encodingparameter set.
 4. The video encoding parameter set estimation system ofclaim 3, wherein the reference set measurement module is configured tomeasure a visual quality of a reference encoding parameter set.
 5. Thevideo encoding parameter set estimation system of claim 1, wherein theoffline encoding parameter set characteristic prediction module includesa specific value difference measurement module configured to measure anencoding time difference of a specific value encoding parameter set. 6.The video encoding parameter set estimation system of claim 5, thespecific value difference measurement module configured to measure avideo quality difference of a specific value encoding parameter set. 7.The video encoding parameter set estimation system of claim 1, whereinthe encoding parameter set search module utilizes candidate encodingparameter set information and information from the offline encodingparameter set characteristic differential module.
 8. The video encodingparameter set estimation system of claim 1, wherein the encodingparameter set search module is configured to: compare an estimatedencoding time to a time constraint; and a further search analysis of avideo quality characteristic of the video encoding parameter set isperformed, conditional upon satisfying the time constraint, wherein thefurther search analysis based upon the objective.
 9. The video encodingparameter set estimation system of claim 8, wherein the encodingparameter set search module is configured to establish a candidateencoding parameter set with a video quality characteristic closet to theobjective as the predicted or estimated optimal video encoding parameterset.
 10. A video encoding parameter set estimation method comprising:performing an offline encoding parameter set characteristic predictionprocess that determines an estimate of a candidate encoding parameterset characteristic; and performing an encoding parameter set searchprocess that identifies a predicted or estimated optimal video encodingparameter set, wherein the encoding parameter set search processincludes: applying a constraint to the candidate encoding parameter setcharacteristic; and determining if candidate encoding parameter setmeets an objective, wherein the determining is performed if theconstraint is satisfied.
 11. The video encoding parameter set estimationmethod of claim 10, wherein the candidate encoding parameter setcharacteristic is an estimated encoding time of the candidate encodingparameter set.
 12. The video encoding parameter set estimation method ofclaim 10, wherein the objective is the best video quality out of aplurality of candidate encoding parameter sets.
 13. The video encodingparameter set estimation method of claim 10, wherein the offlineencoding parameter set characteristic prediction process includes: anoffline encoding characteristic differential process that determinesdifferences in a characteristic of a reference parameter set and acharacteristic of a specific value parameter set; and an encodingparameter set summation process that adds offline encodingcharacteristic differential values together.
 14. The video encodingparameter set estimation method of claim 11, wherein the offlineencoding characteristic differential process includes an analysis of acharacteristic based upon controlled changes to an encoding parameter ofthe specific value parameter set.
 15. The video encoding parameter setestimation method of claim 14, wherein the controlled changes make anencoding parameter in the specific value parameter set encodingdifferent than the encoding parameters in the reference parameter set.16. The video encoding parameter set estimation method of claim 14,wherein one encoding parameter is changed in the specific valueparameter set to be different than the encoding parameters of thereference parameter set while the remaining parameters in both thespecific value parameter set and reference parameter set are the same.17. The video encoding parameter set estimation method of claim 13,wherein a difference between the encoding time of the referenceparameter set and the encoding time of a specific value parameter set isdetermined.
 18. The video encoding parameter set estimation method ofclaim 13, wherein a difference between the video quality of thereference parameter set and the encoding time of a specific valueparameter set is determined.
 19. The video encoding parameter setestimation method of claim 11, wherein if an assigned characteristicvalue does not meet the search constraint the encoding parameter setassociated with assigned value is removed from consideration as anestimated optimal encoding parameter set.
 20. The video encodingparameter set estimation method of claim 11, wherein an encodingparameter search process includes dynamic programming.